ON WEAKLY CONVEX STAR-SHAPED POLYHEDRA 



JEAN-MARC SCHLENKER 

Abstract. Weakly convex polyhedra which are star-shaped with respect to one of their vertices are infinitcs- 
imally rigid. This is partial answer to the question whether every decomposable weakly convex polyhedron is 
infinitesimally rigid. The proof uses a recent result of Izmestiev on the geometry of convex caps. 



1. Introduction 

1.1. The rigidity of convex polyhedra. The rigidity of Euclidean polyhedra has been of interest to geometers 
since Legendre |10j and Cauchy [4] proved that convex polyhedra are globally rigid. This result was an important 
source of inspiration in subsequent geometry for instance for the theory of convex surfaces, and was a key tool 
in Alexandrov's theory of isometric embeddings of polyhedra [T| 1 1 2 j . 

The notion of global rigidity leads directly to the related notion of infinitesimal rigidity; a polyhedron is 
infinitesimally rigid if any non-trivial first-order deformation induces a non-zero variation of the metric on 
one of its faces. Infinitesimal rigidity is important in applications since a structure which is rigid but not 
infinitesimally rigid is likely to be physically unreliable. Although Cauchy's argument can be used to prove 
that convex polyhedra are infinitesimally rigid, this result was proved much later by M. Dehn |8j, by completely 
different methods. 

1.2. Non-convex polyhedra. Cauchy's theorem left open the question of rigidity of non-convex polyhedra, 
until examples of flexible polyhedra were constructed by Connelly [5] . It would however be interesting to know 
a class of rigid polyhedra wider than the convex ones. We say that a polyhedron is weakly convex if its vertices 
are the vertices of a convex polyhedron, and that it is decomposable if it can be cut into convex polyhedra 
without adding any vertex. 

Question 1.1. Let P be a weakly convex, decomposable polyhedron. Is P infinitesimally rigid ? 

This question came up naturally in [16j . where it was proved, using hyperbolic geometry tools, that the result 
is positive if the vertices of P are on an ellipsoid, or more generally if there exists an ellipsoid which contains 
no vertex of P but intersects all its edges. It was proved in [6] that the answer is also positive for two other 
classes of polyhedra: suspensions - which can be cut into simplices with only one interior edge - and polyhedra 
which have at most one non-convex edge, or two non-convex edges sharing a vertex. 

1.3. Main result. Here we extend the result of i6i to a wider class of weakly convex decomposable polyhedra. 
From here on all the polyhedra we consider are triangulated; it is always possible to reduce to that situation by 
adding "flat" edges in non-triangular faces. For a polyhedron P which is star-shaped with respect to a vertex vq, 
we do this subdivision by decomposing all non-triangular faces adjacent vq by adding only diagonals containing 
Vq, so that this refinement of the triangulation of the boundary of P is compatible with a triangulation of the 
interior of P for which all simplices contain Vq. 

Theorem 1.2. Let P be a weakly convex polyhedron, which is star-shaped with respect to one of its vertices. 
Then P is infinitesimally rigid. 

Here by "star-shaped" with respect to vq we mean that the interior of P has a decomposition as the union of 
finitely many non-degenerate simplices, all containing vq as one of their vertices, of disjoint interior, and such 
that the intersection of each with P is a face of both. 

1.4. A refined statement. There is a slightly refined version of Theorem ll.21 giving a better understanding on 
the reasons for which rigidity holds. Let P be star-shaped with respect to a vertex vq, and let P = Si U • • • U Sn 
be a triangulation of P as a union of non-degenerate simplices all containing vq (and having disjoint interior). 
Let ei, ■ • • , e m be the interior edges of this triangulation, i.e., the edges of the Sj which are not contained in 
faces of P. Let U be the length of e^. 
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It is then possible to consider a wider class of (small) deformations of the metric on the interior of P: those 
for which the vary, while the length of the edges of P remain constant. Such a variation of the U determines a 
unique deformation of the metric on the Sj , which can however still be glued isometrically along their common 
faces. Under such a variation, cone singularities might appear along the e^: the angles around those edges might 
become different from 2ir. We call 8i the angle around e,;. 

Definition 1.3. Let 



Note that Ap a priori depends also on the decomposition P = Si U • • • U Sn (and of the labeling of the ej ) . 
It is well-known that Ap is symmetric (see e.g. [5]), this follows from the fact that Ap is minus the Hessian of 
the total scalar curvature of the metrics obtained by varying the U. The following statement is also well known. 

Remark 1.4. P is infinitcsimally rigid if and only if Ap is non-degenerate. 

The proof is elementary: isometric first-order deformations of P correspond precisely to first-order variations 
of the U which do not change, at first order, the 9i. Although the proof of this point requires some care, we do 
not include one here and refer the reader to [HI H] where a similar problem is treated in full details. 

Theorem 1.5. Under the hypothesis of Theorem \1.2\ hp is positive definite. 

1.5. A word on the proof. The proof is only indirectly related to the argument used in |T6], and different 
from those used in [B]. It is based on a recent result of Izmestiev [5], who gives a new proof of Alexandrov's 
theorem on the existence and uniqueness of a polyhedral convex cap with a given induced metric, based on the 
concavity of a geometric function. We slightly extend his argument, to encompass weakly convex "caps" , by 
proving that "removing" a simplex to a (weakly) convex cap actually makes this function "more" concave - a 
point which we found somewhat surprising. We then use a classical projective argument to obtain Theorem 1 1.21 
Theorem 11.51 follows from the same arguments. 



We use a notion of "convex cap" which is a little different from the one used by Izmestiev [9] . We will use a 
different name to avoid ambiguities. In the whole paper we consider a distinguished oriented plane, which can 
for convenience be taken to be the horizontal plane {z = 0}. We call the half-space bounded by this plane 
{z = 0} on the the side of its oriented normal. 

Definition 2.1. Let E C R+, its shadow Sh(E) is the set of points m £ R+ for which there exists a point 
m' G E such that m is contained in the segment joining ml to its orthogonal projection on the horizontal plane 



Definition 2.2. A convex hat is a polyhedral surface H in Rj_ such that 

(1) H is homeomorphic to a disk and has finitely many vertices, 

(2) no point of H is in the shadow of another, 

(3) H is contained in the boundary of the convex hull of Sh(H). 

Note that we do not demand that Sh(H) is convex, and that the projection of H on {z = 0} is not necessarily 
convex. 

Definition 2.3. A weakly convex hat is a polyhedral surface H C R+, satisfying conditions (1) and (2) of 

the previous definition, and such that every vertex of H is an extremal point of the convex hull of Sh(H). 

A convex hat can be obtained by the following procedure. Start from a convex polyhedron P, and apply a 
projective transformation sending one of the vertices, v, to infinity in the vertical direction (towards z — > — oo). 
Then remove all edges and faces adjacent to v. Any convex hat which has as its projection on {z = 0} a convex 
polygon can be obtained in this manner. In the same way, one can start from a weakly convex polyhedron which 
is star-shaped with respect to one of its vertices, say vq, and apply a projective map sending vq to infinity, to 
obtain a weakly convex hat. Any weakly convex hat can be obtained in this manner. 

The proof of Theorem 1 1.21 will follow from the following lemma, using classical arguments relating projective 
transformations to infinitesimal rigidity. 

Lemma 2.4. Let H be a weakly convex hat. Any first- order isometric deformation of H which fixes the heights 
of the boundary vertices is trivial. 




2. Weakly convex hats 



{z = 0}. 
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The proof relies on an extension of the notion of (weakly) convex cap, to allow for cone singularities along 
vertical edges, a device which is common in the field known as "Regge calculus" , or more specifically e.g. in 
[2], [9], or in the last part of [6]. This type of construction has been used successfully also in the contexts of 
hyperbolic polyhedra or circle patterns on surfaces, see e.g. [13 l H4 l \TT \ \3\ flB ] [17] . 

Definition 2.5. A prism is a non-degenerate convex polyhedron P in which is the shadow of a triangle in 



The faces which are neither the bottom or the upper face of P are its vertical faces. It is not difficult to 
check that a prism is uniquely determined, among prisms with the same induced metric on the upper face, by 
the heights of its vertical edges. 

Definition 2.6. A generalized hat is a metric space obtained from a finite set of prisms P±, ■ ■ ■ , P/v by isomet- 
rically identifying some of their vertical faces, so that 

• each vertical face is glued to at most one other, so that singularities occur only at line segments 
corresponding to some vertical edges of the Pj , 

• the prisms containing a given vertical edge are pairwise glued along vertical faces in a cyclic way (with 
either all vertical faces containing the given vertical edges pairwise glued, for an interior edge, or with 
two faces not glued and corresponding to vertical boundary faces of the generalized hat, for a boundary 
edge), 

• under the gluing of two vertical faces, the segments corresponding to the bottom (resp. upper) face of 
the Pi are identified. 

Given a convex or weakly convex hat H, it can be used to construct a generalized hat G by gluing the 
shadows of the faces of H . Moreover it's easy to characterize the generalized hats obtained in this manner. It is 
necessary that the angles around all interior "vertical" edges are equal to 2tt; under this condition, generalized 
hats admit an isometric immersion into R+, with their bottom faces sent to {z — 0}, and a generalized hat G is 
obtained from a (weakly) convex hat H if and only if this image in R 3 is embedded and (weakly) convex. This 
simple construction allows us to consider convex or weakly convex hats as special cases of generalized hats. 

We define a generalized hat to be convex if it is convex at each edge e which is shared by the upper faces of 
two of the prisms Pi and Pj, i.e., if the angles at e of Pi and Pj add up to at most n. It is strictly convex if 
those angles add up to strictly less than tt. 

Given a generalized hat G, one can consider the space Ada of all generalized hats for which the upper 
boundary has the same combinatorics and the same induced metric. It is not difficult to check that M.q is 
parametrized by the heights of the vertical edges hi, ■ ■ ■ , h n . We call M.g,o the subspace of Mg of generalized 
hat having the same boundary heights as G, so that Mg,o is parametrized by the heights of the interior vertical 
edges, hi,-- ■ ,h m . 



We mainly recall in this section results of Izmestiev [9] , adapting the arguments to the proof of Lemma 12.41 
for the special case of convex hats. In the next section it is shown how the argument can be extended to weakly 
convex hats. 

The proof is based on a matrix very similar to the matrix Ap appearing in Definition 11.31 We consider a 
convex hat H , and the corresponding generalized hat G. Since a prism, with given induced metric on its upper 
face, is uniquely determined by the heights of its vertices, elements of Mg,o are uniquely determined by the 
heights of the interior vertices. Conversely, each choice of those heights, close to the heights of the interior 
vertices in H, determines an element of Mg,q- We call ei, • ■ ■ , e m the vertical edges ending at interior points 
of H, (hi)i<i< m their heights, and (0i)i<i< m the angle around them. So the 0i are equal to 2tt for G, but not 
necessarily at other points of Aic.o- 

Lemma 3.1 (Izmestiev [9]). Let 



Then Aq is symmetric and positive definite. 

We only give a brief outline of the proof here. The symmetry of Aq follows from the fact that it is minus the 
Hessian of a natural "total scalar curvature" function appearing in this context, called S in [5]. The coefficients 
of Aq are computed explicitly in [3] (Proposition 4, note that the coefficients given there are minus the ones 
considered here), they are equal to: 
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a,ij = when i ^ j and and ej are not the endpoints of an interior edge of H. 

a,ij = — (cotan(aij) + cota,n(a j^)/lij sin 2 (pij) when i ^ j but e% and ej are the two endpoints of an 
interior edge of H. Here ctij and ctji are the angles between the shadow of the edge of G joining the 
endpoints of a and ej with the two upper faces of G adjacent to that edge, Uj is the length of that 
edge, and is its angle with the vertical. 



a ii ~ ^2ij^i 



It follows from this explicit description that Aq has dominant diagonal, and therefore that it is positive definite. 

Remark II .41 still applies in this context, so that it follows from Lemma 13 . 1 1 that convex hats are infinitesimally 
rigid. 

4. Weakly convex hats are rigid 



The proof of Lemma 12.41 follows from Lemma 13.11 by a simple argument, remarking that (1) it is possible to 
go from a weakly convex hat to a convex hat by adding a finite set of simplices (which are in a specific position 
with respect to the vertical direction) (2) when removing such a simplex, the matrix A defined above becomes 
"more" positive. 

Lemma 4.1. Let H be a weakly convex hat, and let H c be the convex hat which is the union of the upper faces 
of the convex hull of Sh(H) which project orthogonally to {z = 0} as a polygon in the projection of H . There 
exists a finite sequence Hq, ■ • ■ , H p of weakly convex hats in such that 

• Hq = H and H p = H c , 

• for all i 6 {1, ••• ,p}, Hi has the same vertices as i, and Sh(Hi) is obtained from Sh(Hi-i) by 
gluing a simplex Si, 

• the projection of Si on {z = 0} is a quadrilateral. 

Proof. Set Hq := H, and choose a concave edge cq of Hq (which is thus not a boundary edge of Hq) with 
vertices vq and v\. Let / and /' be the faces of H adjacent to eo, and let 113 and V4 be the vertices of / and /' 
opposite to eo- Let S± be the simplex with vertices vq,vi ,V2,Vs, then Si projects to {z = 0} as a quadrilateral. 

We can add to Sh(H) the simplex Si, this yields a polyhedron in which is the shadow of a weakly convex 
hat Hi (which by construction has the same vertices as Ho). 

If Hi is convex, the lemma is proved. Otherwise, Hi has at least one concave edge and one can choose one 
of those edges, say ei, and repeat the construction, adding a simplex S 2 - 

After a finite number of steps the weakly convex hat H p obtained in this way will be convex, because the 
number of simplices that can be added is bounded from above, for instance by the number of Euclidean simplices 
having as vertices some vertices of H. □ 

The next step is to describe in what manner the matrix A associated to a weakly convex hat changes when 
a simplex is removed. We consider a simplex S with vertices v%, V2, i>3, V4 which projects on the plane {z = 0} 
as a quadrilateral. Then the boundary of S is the union of two surfaces, each made by gluing two triangles, 
and each of which has injective projection on {z = 0}: the "lower" surface S-, and the "upper" surface S+, 
with S- C Sh(S+). We suppose for instance that S- is the union of the triangles (1)1,1)3,1)4) and (i>2,i>3,i>4), 
while S+ is the union of (t>i, V2, 1*3) and (vi,V2,Vi). Let hi be the height of Vi over {z = 0}. Any first order 
variation of the hi, 1 < i < 4, determines a first-order displacement of the which preserves the lengths of the 
five segments in S- (including the diagonal), which is unique up to horizontal translation and rotation with 
vertical axis. Similarly a first-order variation of the hi determines a displacement of the V4 which preserves the 
lengths of the five segments in S + . 

Definition 4.2. Let 

where, for heights of the Vi close to the hi, 9f is the angle of the projection of S+ on {z = 0} at the projection 
of Vi, and 6~ is the angle of the projection of S— at the projection of Uj. 

Lemma 4.3. Let H and H' be two weakly convex hats, with Sh(H') obtained by removing from Sh(H) a simplex 
S. Then Ah> is obtained by adding Ms to Ah (with the lines/columns of Ms added to the lines/columns of Ah 
corresponding to the same vertices). 

Proof. This follows from the definitions, since Ah' is equal to A# except that the variation of the curvature 
at the vertical edges ending on the vertices of S are given by the lower surface S- rather than by the upper 
surface S+. □ 
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The interesting point is that M$ is always positive semi-definite, so that adding it to a positive definite 
matrix yields another positive definite matrix. 

Lemma 4.4. For any simplex S projecting on {z = 0} as a quadrilateral, Mg is positive semi- definite of rank 
1. 

Proof. The space of Killing fields in R 3 has dimension 6. It contains a 3-dimensional subspace fixing {z = 0}, 
and therefore acting on S without changing any of the heights. There remains a 3-dimensional vector space 
of Killing fields which do change the heights h{. Each acts by deforming globally S, so that, at each vertex, 
the angles of the projection of the upper and the lower surface change in the same way, and therefore those 
first-order variations of the hi are in the kernel of Ms- So the rank of Ms is at most 1. 

The same argument can be used, conversely, to show that the rank of Ms can not be zero. Otherwise the 
kernel of Ms would have dimension 4, which would mean that there exists a non-trivial first-order deformation 
of S leaving invariant the lengths of all edges in both the upper and lower surfaces, and such that the angles 
of the projections of the upper and lower surface vary in the same way. One could then consider the first-order 
deformations of the upper and of the lower surface, and add a trivial deformation so that they match at all 
four vertices, because the first-order variations of both the heigths of the vertices and the projections of the two 
surfaces on {z = 0} match. This would mean that there is a non-trivial isometric deformation of S, and this is 
well known to be impossible - all simplices are infinitesimally rigid. 

So the signature of Ms is constant over the space of simplices which projects on {z = 0} as quadrilaterals. 
This means that Ms is either positive semi-definite or negative semi-definite for all such simplices. To decide 
which happens, it is sufficient to check for one simplex, for instance a maximally symmetric one. Consider the 
first-order deformations pictured in Figure 1, with the heights of v\ and V2 raised and the heights of V3 and V4 
lowered. 




Figure 1 . A positive deformation of a simplex 

It is easy to check that in this case: 

• 9^ and 9^ decrease: the angles of the projection of the upper surface at the projections of v\ and vi 
decrease, 

• $3 and 9\ increase, 

• 9^ and 9^ increase, 

• #3~ and 9^ decrease, 

It follows that the first-order variation of 9~ — 9f is positive at v% and v-i and negative at U3 and V4, so that 
Ms has at least one positive eigenvalue. So M$ is positive semi-definite. □ 

Lemma 4.5. Let H be any weakly convex hat, then Ah is positive definite. 

Proof. Let H c be the convex hat obtained as the upper boundary of Sh(H). Lemma f4.ll shows that Sh(H) is 
obtained from Sh{H c ) by removing a finite sequence of simplices. But Ah c is positive definite by Lemma 13. II 
and Lemma 14.31 shows that, each time a simplex is removed, the matrix Ah changes by the addition of a 4 x 4 
matrix, which is positive semi-definite by Lemma 03J It follows that Ah is also positive definite. □ 

Proof of Lemma \2.J\ We have already seen that the fact that Ah is non-degenerate implies that H is infinites- 
imally rigid: any isometric first-order deformation of H which fixes the boundary heights is trivial. □ 

5. Projective maps 

The goal of this section is to prove Theorem 11.21 concerning polyhedra which are star-shaped with respect 
to one of their vertices, using Lemma 12.41 which deals with weakly convex hats. The basic idea here is old, 
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going back at least to Darboux [7J and Sauer [15) : infinitesimal rigidity is a property which is invariant under 
projective maps. The particular case of this property which is used here can be stated more precisely as follows. 

Lemma 5.1. Let dq £ M 3 C RP 3 , and let <j> : RP 3 — > RP 3 be a projective transformation sending vq to the 
point at infinity corresponding to the vertical direction in R 3 . There exists a map $ : TR 3 — > PR 3 sending 
(x,v) £ PR 3 to {(j)(x),^ x (v)) £ PR 3 such that: 

• the image by <E> of any Killing vector field in R 3 is a Killing vector field, 

• Killing fields which are infinitesimal rotations of axis containing vq are sent to the translations along 
horizontal directions and the infinitesimal rotations of vertical axis. 

The proof of this Lemma is left to the reader, since it is quite classical. The map ip x can be explicitly 
described as follows: it sends vectors parallel to the direction of Vq to vertical vectors of the same norm, while 
acting on vectors orthogonal to the direction of Vq as the differential of the projective map <f>. 

Proof of Theorem ] Let P be a weakly convex polyhedron which is star-shaped with respect to a vertex i?q. 
Let U be an isometric first-order deformation of P, i.e., the restriction of V to each face of P is a Killing field. 
Adding a global Killing field if necessary, we can assume that the restriction of U to all faces of P containing 
wo is a Killing field fixing Do, i.e., an infinitesimal rotation with axis containing vq. 

Let Q = <fi(P), then Q is an infinite polyhedron with infinite vertical faces corresponding to the faces of P 
containing vq . Applying a vertical translation if necessary, we can suppose that the intersection of Q with R+ 
is of the form Sh(H), where H is a weakly convex hat with one face in its upper boundary corresponding to 
each face of P not containing vq (and conversely). 

Now let V — &(U), then, by Lemma f5.lt the restriction of V to each face of Q is a Killing field, so that V 
is a first-order isometric deformation of Q. Moreover, since the restriction of U to each face of P containing vq 
fixes vq , the restriction of V to the vertical faces of Q are horizontal translations or rotations around a vertical 
axis. So V does not change the heights of the boundary vertices of H. It follows from Lemma \2. 41 that V is a 
trivial deformation - the restriction to Q of a global Killing vector field and therefore, again from Lemma [5~T1 
that U is a trivial deformation of P. So P is infinitesimally rigid. □ 

Note that this argument - along with the results recalled in section 3, but without the need of section 4 - 
gives a direct proof of the infinitesimal rigidity of convex polyhedra. 

Proof of Theorem ] 1.51 Let again Q = 4>(P), so that, applying a vertical translation again if necessary, QnR 3 ^ = 
Sh(H), where H is a weakly convex hat. Let (</>t)tg[o,i] be a one-parameter family of projective transformation, 
chosen such that 

• 0o is the identity, while cf>i = <f>, 

• 4>t(P) is a compact polyhedron in R 3 for all t £ [0, 1). 

Let Pj = 4>t(P), < t < 1. We know by Theorem 1 1.21 that Pt is infinitesimally rigid for all t £ [0, 1). This means 
by Remark ll.4l that Ap 4 has maximal rank, so that the signature of Ap t is constant for t £ [0, 1). 

But a quick look at the definitions shows that Unit— ,1 Ap t = A#, which is positive definite by Lemma l2.41 It 
follows that Ap = Ap is also positive definite. □ 

Acknowledgements. I'm grateful to Bob Connelly, Francois Fillastre and Ivan Izmestiev for useful conversa- 
tions, and for relevant remarks on the first draft of this text. 
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